Partial differential equations (PDEs) are usually solved by numerical methods requiring specific grids and discretisation schemes. Physics-Informed Neural Networks (PINNs) have emerged recently, as a more flexible alternative approach. They are based on feed-forward neural networks, whose parameters are calibrated by minimisation of a loss function combining the residuals of the PDEs, the boundary conditions and possible data. PINNs have several advantages, such as the ability to solve forward, inverse or parametric problems, and integrate data measurements. However, some convergence difficulties have been reported. The aim of this study is therefore to analyse and improve the convergence of PINNs using a multi-criterion optimisation viewpoint. The analysis is performed on a benchmark problem involving the stationary flow in a differentially heated cavity, modelled by the incompressible Navier-Stokes equations with thermal transport. It is demonstrated that PINNs converge slowly but respect the physics, contrary to classical data-fitting approaches. The slow convergence is related to an ill-conditioned Hessian matrix, requiring second-order optimisers. The antagonism between the loss terms related to the residuals of the PDEs and those related to the boundary conditions and data has been characterised and could be a cause of the ill-conditioning. To overcome these difficulties, a multi-criterion approach is then proposed by defining a Nash game to balance the minimisation of the different losses during the training. Results will be presented and analysed, for a simple one-dimensional problem and then for more demanding cases.